Display control apparatus, display control method, and display control program

ABSTRACT

Disclosed herein is a display control apparatus including an image producing part that produces an image of a virtual space to be displayed on a head-mounted display, a display control part that causes the head-mounted display to display thereon the image of the virtual space, and a position selecting part that accepts a selection instruction for a position in the virtual space displayed on the head-mounted display, through an input apparatus used by a user. When the position selecting part accepts the selection instruction for the position from the user, the position selecting part causes the head-mounted display to display thereon an image of the input apparatus, and accepts the selection instruction for the position from a plurality of selectable positions in accordance with an input position relative to a touch pad included in the input apparatus.

CROSS REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Japanese Priority PatentApplication JP 2019-106245 filed Jun. 6, 2019, the entire contents ofwhich are incorporated herein by reference.

BACKGROUND

The present disclosure relates to a display control technique and, inparticular, to a display control apparatus, a display control method,and a display control program that each control display onto ahead-mounted display.

A technique of displaying an image of a virtual space on a head-mounteddisplay that is attached to the head of a user has been provided. In anordinary stationary-type display, a range of a field of view of a userextends also to the outside of the screen of the display. As a result,the user may have difficulty in concentrating on a screen of thedisplay, and a sense of immersion into a video may be insufficient, insome cases. In this regard, when a user wears the head-mounted display,the user consequently views only the video displayed on the head-mounteddisplay, so that the sense of immersion into the world in the video canfurther be enhanced.

SUMMARY

The inventor of the present disclosure has recognized that a moreconvenient display control technique is necessary for allowing a largernumber of users to enjoy a virtual reality using the head-mounteddisplay.

A display control apparatus according to an embodiment of the presentdisclosure includes an image producing part that produces an image of avirtual space to be displayed on a head-mounted display, a displaycontrol part that causes the head-mounted display to display thereon theimage of the virtual space, and a position selecting part that accepts aselection instruction for a position in the virtual space displayed onthe head-mounted display, through an input apparatus used by a user.When the position selecting part accepts the selection instruction forthe position from the user, the position selecting part causes thehead-mounted display to display thereon an image of the input apparatus,and accepts the selection instruction for the position from a pluralityof selectable positions in accordance with an input position relative toa touch pad included in the input apparatus.

Another embodiment of the present disclosure is a display controlmethod. In this method, a display control apparatus produces an image ofa virtual space to be displayed on a head-mounted display, causes thehead-mounted display to display thereon the image of the virtual space,and accepts a selection instruction for a position in the virtual spacedisplayed on the head-mounted display, through an input apparatus usedby a user. When the selection instruction for the position is acceptedfrom the user, the head-mounted display is caused to display thereon animage of the input apparatus, and the selection instruction for theposition is accepted from a plurality of selectable positions inaccordance with an input position relative to a touch pad included inthe input apparatus.

In addition, those that are each formed by converting an optionalcombination of the above constituent elements and the expression of thepresent disclosure among a method, a device, a system, and the like arealso valid as aspects of the present disclosure.

According to the present disclosure, the convenience of a head-mounteddisplay for a user can be improved.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram depicting a usage environment of a display controlsystem according to an embodiment;

FIG. 2 is an outer appearance diagram of a head-mounted displayaccording to the embodiment;

FIG. 3 is a functional configuration diagram of the head-mounteddisplay;

FIGS. 4A and 4B are outer appearance diagrams of an input apparatus;

FIG. 5 is a diagram depicting a configuration of a display controlapparatus;

FIG. 6 is a functional configuration diagram of the display controlapparatus;

FIG. 7 is a diagram depicting an example of a display screen displayedon the head-mounted display;

FIG. 8 is a diagram depicting an example of the display screen displayedon the head-mounted display;

FIG. 9 is an enlarged diagram of a position selection instructionacceptance screen depicted in FIG. 8 ;

FIGS. 10A and 10B are diagrams illustrating other examples of theposition selection instruction acceptance screen;

FIGS. 11A and 11B are diagrams illustrating other examples of theposition selection instruction acceptance screen; and

FIG. 12 is a flowchart depicting a procedure for a display controlmethod according to the embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

In this embodiment, a user interface to accept from a user a selectioninstruction for a position in a virtual space displayed on ahead-mounted display will be described. It is considered that a crosskey, an analog stick, or the like that is included in an input apparatusused by a user is used to accept from the user the selection instructionfor a position from a plurality of selectable positions. In this case,for example, the selection of the position is switched one by one inaccordance with the order determined in advance for each input by thecross key or the analog stick, and the inputting therefore needs to beexecuted repeatedly for many times to select a position distant from theposition that is currently selected. Moreover, in a case where the crosskey is used, when the user desires to select another position in anoblique direction from the position being currently selected, it may bedifficult to know which key should be input. In this embodiment,therefore, to enhance the convenience of the user interface, a touch padincluded in the input apparatus is used, and the selection instructionfor a position is accepted in accordance with the input position on thetouch pad.

However, in a state where the user wears the head-mounted display, theuser may not visually recognize directly the touch pad of the inputapparatus. Accordingly, in a case where the user is not used toinputting onto the touch pad, it is difficult for the user to accuratelyand precisely touch a position on the touch pad that corresponds to adesired position, and a new problem arises that the user consequentlyneeds to input for many times for correction. To solve such a newproblem as above, in this embodiment, an operational feeling that causesthe user to feel as if the user performed the inputting actually seeingthe input apparatus gripped by the user is realized by displaying animage of the input apparatus on the head-mounted display. A userinterface with which the position is easily selected can thereby beprovided, and the convenience for the user can therefore be improved.

FIG. 1 depicts a usage environment of a display control system 1according to the embodiment. The display control system 1 includes adisplay control apparatus 10 that executes a display control program, aninput apparatus 16 to input an instruction by the user into the displaycontrol apparatus 10, an imaging apparatus 14 that images a real spaceon the periphery of the user, and a head-mounted display 100 thatdisplays thereon an image of a virtual space produced by the displaycontrol apparatus 10.

The display control apparatus 10 executes the display control program onthe basis of an instruction input by the input apparatus 16 or thehead-mounted display 100, a position or a posture of the input apparatus16 or the head-mounted display 100, or the like, produces an image ofthe virtual space, and transmits this image to the head-mounted display100.

The head-mounted display 100 is a display apparatus that displays imageson display panels thereof that are positioned in front of the eyes ofthe user, by being worn on the head of the user. The head-mounteddisplay 100 displays an image for the left eye on a display panel forthe left eye and an image for the right eye on a display panel for theright eye separately from each other. These images constitute parallaximages acquired by seeing from the right and the left viewpoints, andrealize a stereoscopic vision. In addition, the user views the displaypanels through each of optical lenses, and the display control apparatus10 therefore supplies parallax image data whose optical strain caused bythe lenses is corrected to the head-mounted display 100.

The head-mounted display 100 displays thereon the image of the virtualspace that is produced by the display control apparatus 10. Moreover,the head-mounted display 100 transmits information relating to an inputby the user onto the input apparatus included in the head-mounteddisplay 100, to the display control apparatus 10. The head-mounteddisplay 100 may be connected to the display control apparatus 10 by awired cable, or may also be connected thereto wirelessly through awireless local area network (LAN) or the like.

The input apparatus 16 has a function of transmitting an instructioninput by the user to the display control apparatus 10. In thisembodiment, the input apparatus 16 is configured as a wirelesscontroller capable of executing wireless communication with the displaycontrol apparatus 10. The input apparatus 16 and the display controlapparatus 10 may also establish wireless connection using a Bluetooth(registered trademark) protocol. In addition, the input apparatus 16 maynot be limited to the wireless controller, but may also be a wiredcontroller that is connected to the display control apparatus 10 througha cable. The input apparatus 16 is driven by a battery, and includes aplurality of buttons, a cross key, an analog stick, a touch pad, and thelike to execute the instruction input for the display control apparatus10. When the user operates the button or the like on the input apparatus16, an instruction input by this operation is transmitted to the displaycontrol apparatus 10 by the wireless communication.

The imaging apparatus 14 is a video camera that includes a chargecoupled device (CCD) imaging element, a complementarymetal-oxide-semiconductor (CMOS) imaging element, or the like, imagesthe real space at a predetermined cycle, and thereby produces a frameimage for each cycle. The imaging apparatus 14 is connected to thedisplay control apparatus 10 through a universal serial bus (USB) oranother interface. The image captured by the imaging apparatus 14 isused to derive the positions and the postures of the input apparatus 16and the head-mounted display 100 in the display control apparatus 10.The imaging apparatus 14 may also be a ranging camera capable ofacquiring a distance or a stereo camera. In this case, the distancebetween the imaging apparatus 14, and the input apparatus 16, thehead-mounted display 100, or the like can be acquired by the imagingapparatus 14.

In the display control system 1 of this embodiment, the input apparatus16 and the head-mounted display 100 each include a light emitting partthat is configured to be able to emit light in a plurality of colors.During the display of an image of the virtual space, the light emittingpart emits light in a color instructed by the display control apparatus10 and is imaged by the imaging apparatus 14. The imaging apparatus 14images the input apparatus 16, produces a frame image, and supplies thisimage to the display control apparatus 10. The display control apparatus10 acquires the frame image and derives position information regardingthe light emitting part in the real space from the position and the sizeof the image of the light emitting part in the frame image. In a casewhere the display control apparatus 10 is a gaming apparatus thatexecutes a game program and that causes the head-mounted display 100 todisplay thereon a game image, the display control apparatus 10 mayhandle the position information as an operation instruction for a game,and may cause the position information to be reflected on the processingfor the game such as controlling a motion of a player's character.

Moreover, the input apparatus 16 and the head-mounted display 100 eachinclude an acceleration sensor and a gyro sensor. Detection valuesdetected by the sensors are transmitted to the display control apparatus10 at a predetermined cycle, and the display control apparatus 10acquires the detection values detected by the sensors and acquires aposition information regarding the input apparatus 16 and thehead-mounted display 100 in the real space. In a case where the displaycontrol apparatus 10 is a gaming apparatus, the display controlapparatus 10 may handle the position information as an operationinstruction for a game, and may cause the position information to bereflected on the processing for the game.

The display control apparatus 10 includes a processing apparatus 11, anoutput control apparatus 12, and a storage apparatus 13. The processingapparatus 11 accepts the operation information input by the user intothe input apparatus 16, and produces an image to be displayed on thehead-mounted display 100. The output control apparatus 12 outputs imagedata produced by the processing apparatus 11 to the head-mounted display100. The storage apparatus 13 stores therein the display controlprogram, data such as images of and shapes in the virtual space, andcontrol parameters, that are used by the processing apparatus 11, andother various types of data.

The function achieved by the output control apparatus 12 may beincorporated in the processing apparatus 11. Specifically, the displaycontrol apparatus 10 may include one processing apparatus 11 or mayinclude the processing apparatus 11 and the output control apparatus 12.In the following, the description will be given regarding the functionof providing the image of the virtual space to the head-mounted display100 collectively as the function of the display control apparatus 10.

The output apparatus 15 displays the image produced by the displaycontrol apparatus 10. The output apparatus 15 may be a televisionreceiver that includes a display and a speaker, or may be a computerdisplay. Because the user views the image using the head-mounted display100, the user wearing the head-mounted display 100 does not need theoutput apparatus 15 while another user can view the display image of theoutput apparatus 15 by preparing the output apparatus 15. The outputcontrol apparatus 12 or the processing apparatus 11 may also cause theoutput apparatus 15 to display thereon the same image as the image thatis currently viewed by the user wearing the head-mounted display 100, ormay also cause the output apparatus 15 to display thereon another image.For example, in such a case that a user wearing the head-mounted display100 and another user play a game together, a game image from theviewpoint of the character for the another user may be displayed on theoutput apparatus 15.

FIG. 2 is an outer appearance diagram of the head-mounted display 100according to the embodiment. The head-mounted display 100 includes amain body part 110, a head contact part 112, and a light emitting part114.

The main body part 110 includes a display, a global positioning system(GPS) unit to acquire the position information, a posture sensor, acommunication apparatus, and the like. The head contact part 112 mayinclude a biological information acquiring sensor capable of measuringthe biological information of the user, such as the body temperature,the pulse, the blood components, the perspiration, the brain wave, andthe cerebral blood flow. The light emitting part 114 emits light in acolor instructed from the display control apparatus 10 as above andfunctions as a criterion for calculating the position of thehead-mounted display 100 in the image captured by the imaging apparatus14.

A camera that captures the eyes of the user may further be provided inthe head-mounted display 100. The line of sight, a move of each of thepupils, a nictitation, and the like of the user can be detected by thecamera mounted on the head-mounted display 100.

The head-mounted display 100 will be described in this embodiment whilethe display control technique of this embodiment is also applicable notonly to the head-mounted display 100 in a narrow sense but also to acase where glasses, a glass-type display, a glass-type camera,headphones, a headset (headphones equipped with a microphone),earphones, earrings, an ear-hanging camera, a hat, a hat equipped with acamera, a hairband, or the like is worn.

FIG. 3 is a functional configuration diagram of the head-mounted display100. The head-mounted display 100 includes an input interface 122, anoutput interface 130, a backlight 132, a communication control part 140,a network adaptor 142, an antenna 144, a storing part 150, a GPS unit161, a wireless unit 162, a posture sensor 164, an external input andoutput terminal interface 170, an external memory 172, a clock part 180,a display apparatus 190, and a control part 160.

In FIG. 3 , the elements described as the functional blocks that executevarious processes can be configured by hardware such as a circuit block,a memory, and another large scale integrated circuit (LSI), orimplemented by software such as a program loaded into the memory, or thelike. Thus, it will be understood by those skilled in the art that thesefunctional blocks may be variously implemented by hardware only, bysoftware only, or by a combination of hardware and software. Thefunctional blocks are not limited to any of these.

The control part 160 is a main processor that processes signals such asan image signal and a sensor signal, orders, and data to output theprocessing results. The input interface 122 accepts an operation signaland a setting signal from an input button and the like, and suppliesthese signals to the control part 160. The output interface 130 receivesthe image signal from the control part 160, and causes the displayapparatus 190 to display thereon an image thereof. The backlight 132supplies backlight to a liquid crystal display included in the displayapparatus 190.

The communication control part 140 transmits data input thereinto fromthe control part 160 to the outside by wired or wireless communicationthrough the network adaptor 142 or the antenna 144. Moreover, thecommunication control part 140 receives data from the outside by wiredor wireless communication through the network adaptor 142 or the antenna144 and outputs the received data to the control part 160.

The storing part 150 temporarily stores therein data, parameters,operation signals, and the like to be processed by the control part 160.

The GPS unit 161 receives the position information from a GPS satelliteand supplies the position information to the control part 160 inaccordance with operation signals from the control part 160. Thewireless unit 162 receives the position information from a wireless basestation and supplies the position information to the control part 160 inaccordance with operation signals from the control part 160.

The posture sensor 164 detects the posture information regarding theorientation, the inclination, and the like of the main body part 110 ofthe head-mounted display 100. The posture sensor 164 is realized byappropriately using a gyro sensor, an acceleration sensor, an angularacceleration sensor, and the like in combination.

The external input and output terminal interface 170 is an interface toconnect the peripheral devices such as a USB controller. The externalmemory 172 is an external memory such as a flash memory.

The clock part 180 sets time information using a setting signal from thecontrol part 160, and supplies time data to the control part 160.

FIGS. 4A and 4B are outer appearance diagrams illustrating the inputapparatus 16. FIG. 4A depicts an outer appearance configuration of anupper face of the input apparatus 16. The user operates the inputapparatus 16 gripping a left-side grip portion 78 b with the user's lefthand and gripping a right-side grip portion 78 a with the user's righthand. A direction key 71, analog sticks 77 a and 77 b, and fourdifferent operation buttons 76 which are input parts are provided on anupper face of a housing of the input apparatus 16. Four buttons 72 to 75have different figures marked with different colors thereon in order toallow distinction thereof from one another. In other words, the circlebutton 72 is marked with a red round mark, the cross button 73 with ablue cross mark, the square button 74 with a purple square mark, and thetriangular button 75 with a green triangle mark. On the upper face ofthe housing, a touch pad 79 is provided in a flat region between thedirection key 71 and the operation buttons 76. The touch pad 79 alsofunctions as a depression type button that is held down by being pressedby the user and moreover, that recovers to its original position whenthe user releases the user's hand therefrom.

A function button 80 is provided between the two analog sticks 77 a and77 b. The function button 80 is used to turn on the power source of theinput apparatus 16 and to concurrently set the communication function ofconnecting the input apparatus 16 and the display control apparatus 10,to be active. After the input apparatus 16 is connected to the displaycontrol apparatus 10, the function button 80 is also used to cause thedisplay control apparatus 10 to display a menu screen.

A SHARE button 81 is provided between the touch pad 79 and the directionkey 71. The SHARE button 81 is used to input an instruction from theuser to an operating system (OS) or system software of the displaycontrol apparatus 10. Moreover, an OPTIONS button 82 is provided betweenthe touch pad 79 and the operation buttons 76. The OPTIONS button 82 isused to input an instruction from the user to an application (the game)executed by the display control apparatus 10. The SHARE button 81 andthe OPTIONS button 82 may be each formed as a push-type button.

FIG. 4B depicts the outer appearance configuration of a side face on therear side of the input apparatus 16. The touch pad 79 extends from theupper face of the housing of the input apparatus 16 on the upper side ofthe side face on the rear side of the housing, and a light emitting part85 that is laterally elongated is provided on the lower side of the sideface on the rear side of the housing. The light emitting part 85includes light-emitting diodes (LEDs) for red (R), green (G), and blue(B), and is lit in accordance with light emission color informationtransmitted from the display control apparatus 10.

On the side face on the rear side of the housing, an upper side button83 a, a lower side button 84 a, an upper side button 83 b, and a lowerside button 84 b are provided at positions in a leftwardly andrightwardly symmetrical relationship in a longitudinal direction. Theupper side button 83 a and the lower side button 84 a are respectivelyoperated by the forefinger and the middle finger of the user's righthand, and the upper side button 83 b and the lower side button 84 b arerespectively operated by the forefinger and the middle finger of theuser's left hand. As depicted, the light emitting part 85 is providedbetween a line of the upper side button 83 a and the lower side button84 a on the right side and a line of the upper side button 83 b and thelower side button 84 b on the left side so as not to be concealed by theforefingers or the middle fingers that operate the buttons, and theimaging apparatus 14 can preferably image the light emitting part 85that is lit. The upper side buttons 83 may be configured as push-typebuttons and the lower side buttons 84 may be configured as trigger-typebuttons that are supported for pivotal motion.

FIG. 5 depicts the configuration of the display control apparatus 10.The display control apparatus 10 includes a main power supply button 20,a power-ON LED 21, a standby LED 22, a system controller 24, a clock 26,a device controller 30, a media drive 32, a USB module 34, a flashmemory 36, a wireless communication module 38, a wired communicationmodule 40, a sub system 50, and a main system 60. These configurationsfunction as the processing apparatus 11, the output control apparatus12, and the storage apparatus 13.

The main system 60 includes a main central processing unit (CPU), amemory and a memory controller that are a main storage apparatus, agraphics processing unit (GPU), and the like. The GPU is mainly used ina computing processes for the display control program. These functionsmay be configured as a system-on-chip and may be formed on one chip. Themain CPU has a function of executing the display control program that isrecorded in an auxiliary storage apparatus 2.

The sub system 50 includes a sub CPU, a memory and a memory controllerthat are a main storage apparatus, and the like, includes no GPU, anddoes not have a function of executing the display control program. Thenumber of circuit gates of the sub CPU is smaller than the number ofcircuit gates of the main CPU, and the operation power consumption ofthe sub CPU is smaller than the operation power consumption of the mainCPU. The sub CPU operates also while the main CPU is in a standby stateand is limited in processing function thereof so as to suppress thepower consumption low.

The main power supply button 20 is an input part through which anoperational input from the user is executed, is provided on the frontface of the housing of the display control apparatus 10, and is operatedto turn on or off the power supply to the main system 60 of the displaycontrol apparatus 10. The power-ON LED 21 is lit when the main powersupply button 20 is turned on, and the standby LED 22 is lit when themain power supply button 20 is turned off.

The system controller 24 detects depression of the main power supplybutton 20 by the user. In a case where the main power supply is turnedoff, when the main power supply button 20 is depressed, the systemcontroller 24 acquires this depression operation as an “on-instruction.”In contrast, in a case where the main power supply is turned on, whenthe main power supply button 20 is depressed, the system controller 24acquires this depression operation as an “off-instruction.”

The clock 26 is a real-time clock, produces date and time information atpresent, and supplies this information to the system controller 24, thesub system 50, and the main system 60.

The device controller 30 is configured as an LSI that executes deliveryand reception of information between devices like a southbridge. Asdepicted, such devices are connected to the device controller 30 as thesystem controller 24, the media drive 32, the USB module 34, the flashmemory 36, the wireless communication module 38, the wired communicationmodule 40, the sub system 50, and the main system 60. The devicecontroller 30 absorbs a difference in electric property and a differencein data transfer speed between the devices, and controls the timing fordata transfer.

The media drive 32 is a drive apparatus which operates a read onlymemory (ROM) medium 44, on which application software of a game or thelike and license information are recorded, loaded thereon to read out aprogram, data, and so forth from the ROM medium 44. The ROM medium 44 isa read-only recording medium such as an optical disc, a magneto-opticaldisc or a Blu-ray disc.

The USB module 34 is a module that is connected to an external deviceusing a USB cable. The USB module 34 may be connected to the auxiliarystorage apparatus and the imaging apparatus 14 using a USB cable. Theflash memory 36 is an auxiliary storage apparatus that constitutes aninternal storage. The wireless communication module 38 communicates bywireless with, for example, the input apparatus 16 in accordance with acommunication protocol such as the Bluetooth (registered trademark)protocol or the Institute of Electrical and Electronics Engineers (IEEE)802.11 protocol. In addition, the wireless communication module 38 maysupport the 3rd Generation digital mobile phone system complying withthe International Mobile Telecommunication 2000 (IMT-2000) prescribed bythe International Telecommunication Union (ITU) or, furthermore, maysupport another generation digital mobile phone system. The wiredcommunication module 40 communicates with an external device by wire andis connected to an external network through, for example, an AP 8.

FIG. 6 is a functional configuration diagram of the display controlapparatus 10. The processing apparatus 11 of the display controlapparatus 10 includes a display control part 311, an instruction inputacquiring part 312, a captured image acquiring part 313, an HMDinformation acquiring part 314, an input apparatus information acquiringpart 315, an image producing part 316, and a position selecting part317. These functional blocks can also be realized in various forms byonly hardware, only software, or a combination of these.

The instruction input acquiring part 312 acquires information relatingto the instruction input by the user accepted by the input apparatus 16or the head-mounted display 100, from the input apparatus 16 or thehead-mounted display 100.

The captured image acquiring part 313 acquires an image captured by theimaging apparatus 14, analyses the acquired image, and calculates thepositions and the postures of the input apparatus 16 and thehead-mounted display 100. The captured image acquiring part 313 acquiresframe images at a predetermined imaging speed (such as, for example, 30frames/second) from the imaging apparatus 14, extracts the images of thelight emitting parts of the input apparatus 16 and the head-mounteddisplay 100 from the frame images, and identifies the position and thesize of each of the images of the light emitting parts in the frameimages. For example, the captured image acquiring part 313 identifiescoordinates of the center of gravity and the radius of the image of alight emitting body in the frame image. The user causes the lightemitting part to be lit in a color that is not likely to be used in theenvironment in which the display control system 1 is used, and therebythe image of the light emitting part can be highly precisely extractedfrom the frame image.

The captured image acquiring part 313 derives position informationregarding the input apparatus 16 and the head-mounted display 100 thatare seen from the imaging apparatus 14, from a position and a size ofthe image of the identified light emitting body. The captured imageacquiring part 313 derives positional coordinates of camera coordinates,from the coordinates of the center of gravity of the image of the lightemitting body and, moreover, derives distance information regarding adistance from the imaging apparatus 14, from the radius of the image ofthe light emitting body. These position coordinates and the distanceinformation constitute the position information regarding the inputapparatus 16 and the head-mounted display 100.

The HMD information acquiring part 314 acquires information relating tothe posture of the head-mounted display 100 from the head-mounteddisplay 100. Moreover, the HMD information acquiring part 314 acquiresinformation relating to the position of the head-mounted display 100from the captured image acquiring part 313. These pieces of informationare delivered to the image producing part 316. The information relatingto the posture of the head-mounted display 100 may be acquired by thecaptured image acquiring part 313 analyzing the captured images of thehead-mounted display 100.

The input apparatus information acquiring part 315 acquires informationrelating to the posture of the input apparatus 16 from the inputapparatus 16. Moreover, the input apparatus information acquiring part315 acquires information relating to the position of the input apparatus16 from the captured image acquiring part 313. These pieces ofinformation are delivered to the image producing part 316 or theposition selecting part 317. The information relating to the posture ofthe input apparatus 16 may be acquired by the captured image acquiringpart 313 analyzing the captured images of the input apparatus 16.

In a case where the input apparatus 16 is not imaged by the imagingapparatus 14 as a result of coming out of the imaging range of theimaging apparatus 14 or being concealed by the body of the user, anobstacle, or the like, the input apparatus information acquiring part315 calculates the position of the input apparatus 16 on the basis ofthe position of the input apparatus 16 acquired in the previous sessionand information relating to the posture of the input apparatus 16acquired thereafter. For example, the input apparatus informationacquiring part 315 may calculate the present position of the inputapparatus 16 by calculating a shift from the position of the inputapparatus 16 acquired in the previous session on the basis of the dataregarding translational acceleration acquired from the accelerationsensor of the input apparatus 16. During the time period in which theinput apparatus 16 is not imaged by the imaging apparatus 14, theposition of the input apparatus 16 is similarly calculated sequentially.In a case where the input apparatus 16 is again imaged by the imagingapparatus 14, the position of the input apparatus 16 sequentiallycalculated using acceleration data may not indicate the accurateposition by accumulation of drift errors, and therefore the position ofthe input apparatus 16 that is newly calculated by the captured imageacquiring part 313 may be taken as the present position of the inputapparatus 16. The same is applied to the head-mounted display 100.

The image producing part 316 produces an image of the virtual space tobe displayed on the head-mounted display 100. The image producing part316 sets a viewpoint position on the basis of the selection instructionfor the position by the user accepted by the position selecting part 317described later, sets a direction of a line of sight on the basis of theposture of the head-mounted display 100, and thereby produces an imageof the virtual space. The image producing part 316 correlates theposture of the head-mounted display 100 and the direction of the line ofsight in the virtual space at a predetermined timing, and thereafterchanges the direction of the line of sight associated with change of theposture of the head-mounted display 100. The user can thereby lookaround in the virtual space by actually moving the head thereof, and cantherefore have an experience that causes the user to feel as if the userwere actually present in the virtual space. The image producing part 316may produce an image of the virtual space from videos captured byimaging apparatuses that are installed at a plurality of positions inthe real space, or may produce an image of the virtual space byrendering shape data regarding a virtual three-dimensional space. Theimage producing part 316 produces a display screen by adding informationrelating to the virtual space, an image to be displayed on thehead-mounted display 100, and the like to the image of the producedvirtual space.

The display control part 311 transmits the display screen produced bythe image producing part 316 to the head-mounted display 100 through thewireless communication module 38 or the wired communication module 40,and causes the head-mounted display 100 to display thereon the displayscreen.

The position selecting part 317 accepts the selection instruction forthe viewpoint position in the virtual space displayed on thehead-mounted display 100, through the touch pad 79 of the inputapparatus 16 used by the user. When the position selecting part 317accepts the selection instruction for the position from the user, theposition selecting part 317 causes the head-mounted display 100 todisplay thereon the image of the input apparatus 16, and accepts theselection instruction of the viewpoint position from a plurality ofselectable viewpoint positions in accordance with an input positionrelative to the touch pad 79 included in the input apparatus 16.

FIG. 7 depicts an example of the display screen displayed on thehead-mounted display 100. In the example in FIG. 7 , a video captured byone of imaging apparatuses arranged at a plurality of positions on asite of a lecture meeting is displayed on the head-mounted display 100.This video is captured in the real world while this video is an exampleof an image of the virtual space in point of the fact that this video isnot a video of the real world that is actually present in front of theuser currently viewing this video.

FIG. 8 depicts an example of the display screen displayed on thehead-mounted display 100. When the user depresses the triangular button75 or the touch pad 79 of the input apparatus 16, the position selectingpart 317 displays a position selection instruction acceptance screen 200that includes an image 210 of the input apparatus 16 and an image of thevirtual space including the plurality of selectable viewpoint positions(hereinafter, also referred to as a “virtual map image 220”) on thehead-mounted display 100, and accepts a selection instruction for theviewpoint position through the touch pad 79 of the input apparatus 16.

FIG. 9 is an enlarged diagram of the position selection instructionacceptance screen 200 depicted in FIG. 8 . The position selecting part317 displays the image 210 of the input apparatus 16 on the head-mounteddisplay 100 on the basis of the relative positions of the inputapparatus 16 and the head-mounted display 100. Moreover, the positionselecting part 317 changes the posture of the image 210 of the inputapparatus displayed on the head-mounted display 100 matching with theposture of the input apparatus 16. The user wearing the head-mounteddisplay 100 may not visually recognize the input apparatus 16 in thereal world. However, by displaying the image 210 of the input apparatus16 at the position at which the input apparatus 16 is to be viewed bythe user when the user does not wear the head-mounted display 100, theuser can easily perform input onto the touch pad 79 of the inputapparatus 16.

The position selecting part 317 displays the virtual map image 220 thatincludes the plurality of selectable viewpoint positions, superimposingthis image on a touch pad 211 in the image 210 of the input apparatus 16displayed on the head-mounted display 100 or in the vicinity of thetouch pad 211. This virtual map image 220 may be a planar view imageacquired by seeing the virtual space from a predetermined direction, ormay be an image acquired by setting a predetermined viewpoint positionand a predetermined direction of the line of sight, and therebyrendering the virtual space. In the example in FIG. 9 , the planar viewimage acquired by seeing the virtual space from above is displayed asthe virtual map image 220.

The position selecting part 317 displays a marker 221 indicating theplurality of selectable viewpoint positions, a marker 222 indicating theviewpoint position currently set, and a marker 223 indicating theviewpoint position currently selected.

The position selecting part 317 displays a pointer 212 indicating aninput position relative to the touch pad 79 at a position thatcorresponds to the touch pad 211 in the image 210 of the input apparatus16 displayed on the head-mounted display 100.

When the user depresses the circle button 72 or the touch pad 79, theposition selecting part 317 determines the currently selected viewpointposition and notifies the image producing part 316 of this position. Theimage producing part 316 newly sets the viewpoint position notified offrom the position selecting part 317 and thereby produces an image ofthe virtual space acquired by seeing the virtual space from the newviewpoint position. Before determining the viewpoint position, theposition selecting part 317 may also display a preview image of thevirtual space acquired by seeing the virtual space from the viewpointposition currently selected, in the vicinity of the image 210 of theinput apparatus 16. When the position selecting part 317 determines theviewpoint position, the position selecting part 317 may also cause theinput apparatus 16 to vibrate.

The user can select the viewpoint position visually recognizing theinput position relative to the touch pad 79 by providing such a userinterface as above, and the convenience for the user can therefore beimproved. Even in a case where the user touches a position differentfrom the position on the touch pad 79 that corresponds to a desiredviewpoint position, the user can easily select the desired viewpointposition by referring to the pointer 212 indicating the input positionand the marker 223 indicating the selected viewpoint position, slidingthe finger toward the position that corresponds to the desired viewpointposition, and thereby tracing on the touch pad 79. The image 210 of theinput apparatus 16 and the virtual map image 220 are displayed atpositions at hand of the user, and the image of the virtual spacecurrently viewed is not obstructed and the user can change the viewpointcontinuing the viewing.

The plurality of selectable viewpoint positions is discretely providedin the example of this embodiment while the technique of this embodimentis also applicable to a case where the viewpoint position cancontinuously be varied. According to the technique of this embodiment,even in a case where many discrete viewpoint positions are prepared, oreven in a case where the viewpoint position can continuously be varied,the selection instruction for the viewpoint position can easily beaccepted without causing the user interface for accepting the selectioninstruction to be more complicated.

The position selecting part 317 may classify the plurality of selectableviewpoint positions into a plurality of groups and may accept theselection instruction for the viewpoint position from the viewpointpositions that belong to a group in accordance with an inputting modeonto the touch pad 79. For example, the plurality of viewpoint positionsdepicted in FIG. 9 may be classified into a group of the viewpointpositions on the side of the audience seats and a group of the viewpointpositions on the stage, one group may be selected therefrom inaccordance with the kind of the finger that inputs onto the touch pad79, the number thereof, and the like, and a selection instruction forthe viewpoint position may be accepted in accordance with the inputposition from the viewpoint positions that belong to the selected group.The position selecting part 317 may refer to the captured image acquiredby the captured image acquiring part 313 and may thereby decide theinputting mode onto the touch pad 79. For example, the positionselecting part 317 refers to the captured image, and then determines thekind of the finger that inputs onto the touch pad 79 and the numberthereof. The position selecting part 317 may cause the viewpointpositions on the side of the audience seats to be selectable in a casewhere the inputting is executed by a finger of the right hand, whilecausing the viewpoint positions on the stage to be selectable in a casewhere the inputting is executed by a finger of the left hand. Theposition selecting part 317 may cause the viewpoint positions on theside of the audience seats to be selectable in a case where theinputting is executed by a forefinger, while causing the viewpointpositions on the stage to be selectable in a case where the inputting isexecuted by a middle finger. The position selecting part 317 may causethe viewpoint positions on the side of the audience seats to beselectable in a case where the inputting is executed by one finger,while causing the viewpoint positions on the stage to be selectable in acase where the inputting is executed by two fingers. In a case where thetouch pad 79 is an electrostatic capacitance touch pad capable ofdetecting change of the electrostatic capacitance, the positionselecting part 317 may decide the inputting mode onto the touch pad 79on the basis of the change of the electrostatic capacitance detected bythe touch pad 79. Also in this case, the mode such as the kind, thenumber, or the like of the finger, the nail, or the like used forinputting onto the touch pad 79 can also be decided, and the group ofthe viewpoint positions can therefore be selected in accordance with theresult of the decision.

The position selecting part 317 may divide the virtual space into aplurality of regions, and may switch the region in accordance with theinputting mode onto the touch pad 79. For example, the user may scrollthe region using a swipe operation executed by two fingers, a swipeoperation executed pushing down both sides of the touch pad 79, or thelike. Alternatively, the user may scroll the region according to changeof the line of sight of the user, an operation by the direction key 71or the analog sticks 77 of the input apparatus 16, or the like.Moreover, in a case where lectures, performances, and the like on aplurality of sites can be viewed, the user may scroll the region on thesame one site by a swipe operation in the up-and-down direction and mayswitch the site to another site by a swipe operation in theright-and-left direction.

The position selecting part 317 may accept an selection instruction fora position by the direction key 71 or the analog stick 77 included inthe input apparatus 16. The convenience for the user can further beimproved by enabling the viewpoint positions the to be selected usingnot only the touch pad 79 but also the direction key 71, the analogstick 77, and the like.

The position selecting part 317 may accept the selection instruction forthe position from a terminal apparatus used by the user, or the like. Inthis case, the position selecting part 317 receives from the terminalapparatus an operation input by the user accepted through the touch padby the terminal apparatus capable of communicating with the displaycontrol apparatus 10. When the position selecting part 317 accepts anoperation of, for example, tracing on the touch pad with a finger so asto swirl, the position selecting part 317 may display the positionselection instruction acceptance screen 200 on the head-mounted display100 and may accept the selection instruction for the viewpoint positionin accordance with the input position relative to the touch pad.

The user interface to accept the selection instruction for the viewpointposition used when the image of the virtual space is produced, from theuser, has been described in the above example while the technique ofthis embodiment is also applicable to a case where a selectioninstruction for an optional position such as a position to arrange ormove a character, an object, or the like in the virtual space isaccepted from the user.

The planar view image of the overall virtual space is displayed as thevirtual map image 220 in the position selection instruction acceptancescreen 200 in the example depicted in FIG. 8 and FIG. 9 , while theposition selecting part 317 may display an image of a region of aportion of the virtual space and may accept a selection instruction fora position that is included in the region of the virtual space currentlybeing displayed. For example, the position selecting part 317 maydisplay the image of a region of a portion of the game field as thevirtual map image 220 and may accept a selection instruction for theposition to be a move destination for a character or the like operatedby the user, in the region displayed as the virtual map image 220. Inthis case, the position selecting part 317 may be able to change theregion of the game field to be displayed in accordance with aninstruction from the user, the progress status of the game, or the like.The instruction to change the game field to be displayed may be acceptedby the direction key 71, the analog stick 77, the touch pad 79, or thelike of the input apparatus 16.

FIGS. 10A and 10B depict other examples of the position selectioninstruction acceptance screen 200. In the example depicted in FIG. 10A,a region in the vicinity of the present position of a character 213operated by the user, of the game field, is displayed as the virtual mapimage 220. The position selecting part 317 displays an image of at leasta portion of the game field and the image 210 of the input apparatus ina superimposed manner, and accepts a selection instruction for aposition included in the region of the game field displayed beingsuperimposed on the touch pad 211 of the image 210 of the inputapparatus. In other words, the position selecting part 317 accepts theselection instruction for the position in the game field displayed atthe position of the touch pad 211 that corresponds to the input positionrelative to the touch pad 79. The game control part that executes thegame program moves the character 213 to the position in the game fieldaccepted by the position selecting part 317. As depicted in FIG. 10B,the user can optionally select the position to be the move destinationfor the character 213 changing the region of the game field displayedbeing superimposed on the touch pad 211 by scrolling the region of thegame field displayed as the virtual map image 220.

FIGS. 11A and 11B depict other examples of the position selectioninstruction acceptance screen 200. The region being superimposed on thetouch pad 211 in the image 210 of the input apparatus is changed byscrolling the image of the game field in the examples depicted in FIGS.10A and 10B while, in the examples depicted in FIGS. 11A and 11B, theuser changes the display position of the image 210 of the inputapparatus by changing the position of the input apparatus 16, and theregion of the game field superimposing on the touch pad 211 is set to bechangeable. At this time, the display position of the virtual map image220 may be fixed in advance.

The examples depicted in FIGS. 10A and 10B and the examples depicted inFIGS. 11A and 11B may be combined with each other. In other words, theregion of the game field in which its position is selectable may be setto be changeable by each of both of scrolling the game field and movingthe input apparatus 16.

FIG. 12 is a flowchart depicting a procedure for a display controlmethod according to the embodiment. When the user depresses thetriangular button 75 or the touch pad 79 of the input apparatus 16, theposition selecting part 317 displays the image 210 of the inputapparatus 16 and the virtual map image 220 including the plurality ofselectable viewpoint positions, on the head-mounted display 100 (S100),and accepts the selection instruction for the viewpoint position throughthe touch pad 79 of the input apparatus 16 (S102). When the userdepresses the circle button 72 or the touch pad 79, the positionselecting part 317 determines the viewpoint position currently beingselected (S104) and notifies the image producing part 316 of thisdetermination. The image producing part 316 produces the image of thevirtual space seen from the new viewpoint position, causing the notifiedviewpoint from the position selecting part 317 to reflect thereon(S106).

In the foregoing description, the present disclosure has been describedon the basis of the embodiment. This embodiment is merely illustrative,and it is understood for those skilled in the art that combinations ofconstituting elements and processes of the embodiment can be modified invarious ways and that such modifications are also within the scope ofthe present invention.

The images for monocular vision are displayed on the display apparatus190 of the head-mounted display 100 in the above example, while imagesfor binocular stereoscopic view may be displayed in another example.

What is claimed is:
 1. A display control apparatus comprising: an imageproducing part that produces an image of a virtual space to be displayedon a head-mounted display; a display control part that causes thehead-mounted display to display thereon the image of the virtual space;a position selecting part that accepts a selection instruction for aposition in the virtual space displayed on the head-mounted display,through an input apparatus used by a user, wherein when the positionselecting part accepts the selection instruction for the position fromthe user, the position selecting part causes the head-mounted display todisplay thereon an image of the input apparatus, and accepts theselection instruction for the position from a plurality of selectablepositions in accordance with an input position relative to a touch padincluded in the input apparatus; an input apparatus informationacquiring part that acquires information indicating a position of theinput apparatus used by the user, wherein the position selecting partdisplays the image of the input apparatus at a position determined on abasis of a relative position of the input apparatus to the head-mounteddisplay in a display screen of the head-mounted display.
 2. The displaycontrol apparatus according to claim 1, wherein the position selectingpart accepts the selection instruction for a viewpoint position usedwhen the image producing part produces the image of the virtual space,and the image producing part sets the viewpoint position at a positionaccepted by the position selecting part, and produces the image of thevirtual space.
 3. The display control apparatus according to claim 1,wherein the position selecting part displays a pointer that indicatesthe input position relative to the touch pad, at a position thatcorresponds to a touch pad in the image of the input apparatus displayedon the head-mounted display.
 4. The display control apparatus accordingto claim 1, wherein an image of the virtual space that includes theplurality of selectable positions is displayed, being superimposed onthe touch pad or in a vicinity of the touch pad in the image of theinput apparatus displayed on the head-mounted display.
 5. The displaycontrol apparatus according to claim 4, wherein the image of the virtualspace is a planar view image that is acquired by seeing the virtualspace from a predetermined direction, or an image that is acquired bysetting a predetermined viewpoint position and a predetermined directionof a line of sight and rendering the virtual space.
 6. The displaycontrol apparatus according to claim 4, wherein the position selectingpart displays a marker that indicates the plurality of selectablepositions superimposing the marker on the image of the virtual space. 7.The display control apparatus according to claim 4, wherein when animage of a region of a portion of the virtual space is displayed, theposition selecting part accepts the selection instruction for theposition included in the displayed region of the virtual space, and isable to change the displayed region of the virtual space.
 8. The displaycontrol apparatus according to claim 4, wherein the position selectingpart displays an image of at least a portion of the virtual space andthe image of the input apparatus in a superimposed manner, and accepts aselection instruction for a position included in the region of thevirtual space displayed being superimposed on the touch pad in the imageof the input apparatus.
 9. The display control apparatus according toclaim 8, wherein the position selecting part allows the region of thevirtual space displayed being superimposed on the touch pad in the imageof the input apparatus to be changeable by changing a display positionof the image of the input apparatus in accordance with change of theposition of the input apparatus.
 10. The display control apparatusaccording to claim 1, wherein the position selecting part classifies theplurality of selectable positions into a plurality of groups, andaccepts a selection instruction for a position from the positions thatbelong to the group in accordance with an inputting mode onto the touchpad.
 11. The display control apparatus according to claim 10, furthercomprising: a captured image acquiring part that acquires an imageincluding the input apparatus captured by an imaging apparatus, whereinthe position selecting part refers to a captured image acquired by thecaptured image acquiring part, and decides an inputting mode onto thetouch pad.
 12. The display control apparatus according to claim 10,wherein the touch pad is able to detect change of electrostaticcapacitance, and the position selecting part determines the inputtingmode onto the touch pad on a basis of the change of the electrostaticcapacitance detected by the touch pad.
 13. The display control apparatusaccording to claim 1, wherein the position selecting part accepts theselection instruction for the position by a direction key or an analogstick that is included in the input apparatus.
 14. A display controlmethod, comprising: by a display control apparatus, producing an imageof a virtual space to be displayed on a head-mounted display; causingthe head-mounted display to display thereon the image of the virtualspace; and accepting a selection instruction for a position in thevirtual space displayed on the head-mounted display, through an inputapparatus used by a user, wherein, when the selection instruction forthe position is accepted from the user, the head-mounted display iscaused to display thereon an image of the input apparatus, and theselection instruction for the position is accepted from a plurality ofselectable positions in accordance with an input position relative to atouch pad included in the input apparatus; acquiring informationindicating a position of the input apparatus used by the user; anddisplaying the image of the input apparatus at a position determined ona basis of a relative position of the input apparatus to thehead-mounted display in a display screen of the head-mounted display.15. A non-transitory computer readable medium having stored thereon adisplay control program for a computer, comprising: an image producingpart that produces an image of a virtual space to be displayed on ahead-mounted display; a display control part that causes thehead-mounted display to display thereon the image of the virtual space;and a position selecting part that accepts a selection instruction for aposition in the virtual space displayed on the head-mounted display,through an input apparatus used by a user, wherein when the positionselecting part accepts the selection instruction for the position fromthe user, the position selecting part causes the head-mounted display todisplay thereon an image of the input apparatus, and accepts theselection instruction for the position from a plurality of selectablepositions in accordance with an input position relative to a touch padincluded in the input apparatus; an input apparatus informationacquiring part that acquires information indicating a position of theinput apparatus used by the user, wherein the position selecting partdisplays the image of the input apparatus at a position determined on abasis of a relative position of the input apparatus to the head-mounteddisplay in a display screen of the head-mounted display.